Digital signal processing apparatus and method and providing medium

ABSTRACT

In case of decoding by a software process, in the process  1 , video data of a decoded frame corresponding to a management ID stored at the head of an output FIFO  4  at its time point is read out and outputted. In the process  2 , audio data is decoded. In the process  3 , the video data is decoded. The decoded video data is stored in a video frame buffer and its management ID is stored in the output FIFO  4  in the outputting order. When the process  3  cannot be finished, the process  3  is interrupted during the process and the image of the frame stored at the head of the output FIFO  4  is outputted. After that, when the process  2  is finished, the decoding process of the process  3  which was temporarily interrupted is restarted. In case of performing the encoding by the software process, an encoding amount of the video data as a processing target of the encoding is predicted. Subsequently, whether an empty capacity enough to store the data of the predicted encoding amount exists in a code buffer or not is discriminated. If it is determined that there is the empty capacity, the encoding is started and the video data in which the encoding was finished is deleted from the video frame buffer. The encoding is temporarily interrupted when a reading process or the like of the video data is performed.

TECHNICAL FIELD

The invention relates to digital signal processing apparatus and methodand providing medium. More particularly, the invention relates todigital signal processing apparatus and method for encoding or decodingvideo data and audio data by a software process and relates to aproviding medium.

BACKGROUND ART

In recent years, a system for transmitting digital video data anddigital audio data of a TV program through a satellite has been beingspread. In such a system, to compress a data amount, the digital videodata and digital audio data are encoded by, for example, an MPEG (MovingPicture Experts Group) system and transmitted.

In an encoding apparatus by a software process, an encoding is performedsynchronously with an input period of video data. The input period hasbeen defined by various standards. For example, in the NTSC (NationalTelevision System Committee) system, it is set to 33.36 msec. Themaximum processing time (encoding time) allocated to one image isrestricted by the input period of the video data which has beendetermined by such a standard. In other words, the inputted video datahas to be encoded and the process has to be finished within the inputperiod. The video data encoded in this manner is recorded to apredetermined recording medium or the like or transmitted through asatellite.

A receiving apparatus for receiving digital data transmitted through thesatellite decodes received data by the MPEG system. In case of decodingthe digital video data by a software process, it is necessary tosuccessively execute a plurality of processes which are necessary fordecoding. To output in a real-time manner, it is necessary to completeall of the processes necessary for decoding within a time which ismatched with a period that is required for output. For example, in caseof the NTSC system, since a frame rate is equal to 30 frames/sec, oneperiod is equal to 33.36 msec. It is, therefore, necessary to decode thedigital video data of one frame within 33.36 msec.

In dependence on a situation of a stream which is inputted or asituation of a video image which is decoded, a time necessary forexecuting those processes fluctuates. For example, in case of a videoimage of the MPEG system, a time necessary for a decoding process of avariable length code changes depending on a bit rate of a bit stream. Aprocessing time of a motion compensation also changes depending on akind of picture coding type such as I picture, B picture, or P pictureor depending on a difference of motion compensation precision such ashalf pel or full pel. Further, it also takes time to perform a processsuch as decoding or demultiplexing of the digital audio data as well asthe video image. Further, a time which is consumed by the OS formanaging the whole processes also fluctuates.

Therefore, in case of decoding by the software process by using apersonal computer, when the decoding process cannot be performed withinan output period, the process of a part of data is skipped and an outputis decimated, thereby maintaining real-time performance.

As mentioned above, according to the conventional encoding apparatus,since the encoding is performed synchronously with the input period ofthe video data, the encoding has to be completed at the maximumprecision within the input period for each image. The precision uponencoding depends on a similarity between the image before encoding andthe image obtained by decoding the encoded image. The precision alsodepends on the processing time for encoding, encoding method, or thelike.

In the case where the encoding process is changed on the time base (forexample, MPEG (Moving Picture Experts Group) 2) or in the case whereeach image has to be encoded at the closest precision as possible (toprevent a situation such that when the precision of the encoding differson the time base, the decoded image flickers and becomes hard to beseen), an encoding apparatus has to be designed by setting it so thatthere is an enough time for the encoding process in order to raise theencoding-precision.

The encoding apparatus by the software process has to perform a controlof an input, an output, and the like of the video data in addition tothe encoding process, and those processes have to be simultaneouslyperformed by a built-in processor. There is, consequently, a problemsuch that it is difficult to calculate the longest processing time whichis allocated to the encoding process due to a relation with theprocesses other than the encoding process. Thus, there is also a problemsuch that a vain time during which the processor executes no process iscaused. As mentioned above, in case of allowing one processor to performa plurality of processes, there is a problem such that the processorneeds to have a high processing ability and the costs rise.

The invention is made in consideration of such a situation and it is anobject of the invention to enable encoding at high precision to beperformed even by a processor of a low processing ability by allowing anencoding process of one image to be executed while preferentiallyexecuting other processes.

For example, in a dedicated receiving apparatus for receiving asatellite broadcasting, even in case of decoding a reception signal by asoftware process, the process for decimating an output by skipping aprocess of a part of data in order to maintain real-time performanceshould not be permitted in consideration of the fact that such anapparatus is a dedicated apparatus for decoding a digital video signalwhich is inherently transmitted through a satellite.

In order to enable the decoding to be completed in a short time by thesoftware process, therefore, there are problems such that a processorhaving a very high processing ability is necessary and the costs arehigh.

The invention is made in consideration of such a situation and intendsto enable a decoding process to be performed at low costs by a softwareprocess by using a processor having an ordinary processing ability.

DISCLOSURE OF INVENTION

According to claim 1, there is provided a digital signal processingapparatus comprising: input means for inputting an encoded digitalsignal; decoding means constructed by software for decoding the digitalsignal inputted by the input means; first storing means for storing thedigital signal decoded by the decoding means by an amount correspondingto a plurality of access units; and managing means for managing anoutputting order of the digital signal of the access units stored in thefirst storing means by an FIFO format.

According to claim 3, there is provided a digital signal processingmethod comprising: an input step of inputting an encoded digital signal;a decoding step of decoding the digital signal inputted by a process inthe input step by software; a first storing step of storing the digitalsignal decoded by a process in the decoding step by an amountcorresponding to a plurality of access units; and a managing step ofmanaging an outputting order of the digital signal of the access unitsstored by a process in the first storing step by an FIFO format.

According to claim 4, there is provided a providing medium for providinga computer-readable program for allowing a digital signal processingapparatus to execute processes, wherein the processes comprise: an inputstep of inputting an encoded digital signal; a decoding step of decodingthe digital signal inputted by a process in the input step by software;a first storing step of storing the digital signal decoded by a processin the decoding step by an amount corresponding to a plurality of accessunits; and a managing step of managing an outputting order of thedigital signal of the access units stored by a process in the firststoring step by an FIFO format.

According to claim 5, there is provided a digital signal processingapparatus comprising: input means for inputting video data; firststoring means for storing the video data inputted by the input means;predicting means for predicting a data amount at the time when the videodata stored in the first storing means is encoded; encoding means forencoding the video data stored by the first storing means; and secondstoring means for storing the video data encoded by the encoding means,wherein the encoding by the encoding means is executed if it isdetermined that the data of the amount predicted by the predicting meanscan be stored in the second storing means, and the encoding isinterrupted while the video data is being inputted by the input meansand is being processed.

According to claim 6, there is provided a digital signal processingmethod comprising: an input step of inputting video data; a firststoring step of storing the video data inputted by the input step; apredicting step of predicting a data amount at the time when the videodata stored in the first storing step is encoded; an encoding step ofencoding the video data stored by the first storing step; and a secondstoring step of storing the video data encoded by the encoding step,wherein the encoding in the encoding step is executed in the case wherethe data of the amount predicted by the predicting step can be stored bya process in the second storing step, and the encoding is interruptedwhile the video data is being inputted by the input step and beingprocessed.

According to another embodiment of the invention, there is provided aprogram for a medium, wherein the program comprises: an input step ofinputting video data; a first storing step of storing the video datainputted by the input step; a predicting step of predicting a dataamount at the time when the video data stored in the first storing stepis encoded; an encoding step of encoding the video data stored by thefirst storing step; and a second storing step of storing the video dataencoded by the encoding step, and the encoding in the encoding step isexecuted if it is determined that the data of the amount predicted bythe predicting step can be stored by a process in the second storingstep, and the encoding is interrupted while the video data is beinginputted by the input step and being processed.

In the digital signal processing apparatus according to claim 1, thedigital signal processing method according to claim 3, and the providingmedium according to claim 4, the stored outputting order of the digitalsignal of the access units is managed by the FIFO format.

In the digital signal processing apparatus according to claim 5, thedigital signal processing method according to claim 6, and the mediumaccording to another embodiment of the invention, the data amount at thetime when the inputted video data is encoded is predicted, the videodata is encoded, the encoded video data is stored, the encoding isperformed when it is determined that the data of the predicted amountcan be stored, and the encoding is interrupted while the video data isbeing inputted and processed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a constructional example of a digitalsignal processing apparatus (decoder) to which the invention is applied.

FIG. 2 is a diagram showing a construction of a video frame buffer 3 inFIG. 1.

FIG. 3 is a diagram showing a construction of an output FIFO 4 in FIG.1.

FIG. 4 is a timing chart for explaining the operation of the apparatusof FIG. 1.

FIG. 5 is a flowchart for explaining the operation of a process 1 inFIG. 4.

FIG. 6 is a flowchart for explaining the operation of a process 3 inFIG. 4.

FIG. 7 is a block diagram showing a constructional example of a digitalsignal processing apparatus (encoder) to which the invention is applied.

FIG. 8 is a diagram for explaining a video frame buffer 13.

FIG. 9 is a flowchart for explaining a writing process of video datainto the video frame buffer 13.

FIG. 10 is a flowchart for explaining an encoding.

FIG. 11 is a diagram for explaining processes within a range from aninput of the video data to an output.

FIG. 12 is a diagram showing a sequel to FIG. 11.

FIG. 13 is a diagram for explaining media.

BEST MODE FOR CARRYING OUT THE INVENTION

A digital signal processing apparatus for decoding by a software processwill be first described. FIG. 1 shows a constructional example of adigital signal processing apparatus to which the invention is applied. Adecoding program for performing a decoding process of an inputtedstream, a processing program for controlling each section, and the likehave been stored in a program memory 2. A CPU 1 properly reads out theprogram stored in the program memory 2 through a bus 7 and executes it.A video frame buffer 3 is a storing apparatus for temporarily storingdecoded video data and has a capacity to hold video data of a pluralityof frames.

An output FIFO (First In First Out) 4 stores management IDs of buffers 1to N in which images of frames to be outputted have been stored in theoutputting order. A stream input interface (I/F) 5 executes an interfaceprocess for inputting a transport stream which is transmitted through asatellite or the like and has been encoded by, for example, the MPEGsystem. A display controller 6 executes a process for outputting thedecoded video data to a display (not shown) and displaying it.

As shown in FIG. 2, the video frame buffer 3 has the buffers 1 to N asareas for storing the video data of N frames. It is not always necessaryto define those buffers into continuous address areas. Each buffer isused in order of the allocation for a decoding output, the holding of adecoding result, and the output of the data. The buffer in which theoutput has been finished is used again as a buffer for decoding output.Further, all of the buffers are equivalently used and are not used foran application in which a specific buffer is limited.

The output FIFO 4 can be also formed in the video frame buffer 3.

FIG. 3 shows a principled construction of the output FIFO 4. The outputFIFO 4 manages the order of the buffers for outputting the decodingresult and a delay that is caused before the output. Management IDs orpointers of the buffers 1 to N of the video frame buffer 3 shown in FIG.2 are inputted and stored in the output FIFO 4 in the outputting order.

The operation of the apparatus shown in FIG. 1 will now be describedwith reference to a timing chart of FIG. 4. In FIG. 4, each of time t1to time t4 indicates a timing for outputting the decoded video data.That is, each interval between time t1 and time t4 is set to a period(33.36 msec) of one frame. The CPU 1 executes a process 1 at the timingof time t1 in accordance with a program read out from the program memory2. When the process of the process 1 is finished, the CPU 1 subsequentlyexecutes a process 2. When the process of the process 2 is completed,the CPU 1 subsequently executes a process 3. In this manner, regardingthe priorities of the processes, the process 1 is set to the highestpriority, the process 2 is set to the second highest priority, and theprocess 3 is set to the lowest priority, respectively.

The process 1 will now be described with reference to a flowchart ofFIG. 5. In step S11, the CPU 1 obtains the ID of the buffer to which thedata is outputted next from the output FIFO 4. In step S12, the CPU 1reads out video data corresponding to the ID obtained in step S11 fromthe video frame buffer 3 and outputs it to the display controller 6 viathe bus 7. The display controller 6 outputs the inputted data to thedisplay (not shown) and allows it to be displayed.

For example, “A” has been held as a management ID at the head in theoutput FIFO 4 at time t1 shown in FIG. 4. In step S11, the CPU 1 readsout “A” as a management ID from the output FIFO 4, reads out the videodata of the buffer corresponding to the management ID “A” among thebuffers 1 to N in the video frame buffer 3, and outputs it.

The CPU 1 subsequently executes the process 2. For example, the process2 is a decoding process of audio data. The processed audio data isgenerated from speakers (not shown) synchronously with a predeterminedtiming signal which is outputted from the display controller 6.

When the process 2 is finished, the CPU 1 subsequently executes theprocess 3. The details of the process 3 will now be described withreference to a flowchart of FIG. 6.

First, in step S21, the CPU 1 executes an allocating process of thebuffer for the next decoding output. That is, the empty buffer (whichhas already been read out) among the buffers 1 to N of the video framebuffer 3 shown in FIG. 2 is allocated as a buffer for storing thedecoded video data. In next step S22, the CPU 1 decodes the video dataof one frame inputted from the stream input interface 5 by the decodingprogram read out from the program memory 2 by the MPEG system. Thedecoded data is stored in the buffer allocated in step S21. Further, instep S23, the CPU 1 registers the ID of the buffer to be outputted (readout) next into the output FIFO 4.

At a timing between time t1 and time t2 in FIG. 4, the decoding processof the frame in which the management ID is set to D has been performedby the process 3. Therefore, the CPU 1 registers D as a management IDinto the output FIFO 4. In the example of FIG. 4, the process 3 can becompleted in a period of time from time t1 to time t2.

On the other hand, for a period of time from time t2 to time t3, aftervideo data of a frame in which a management ID is set to B was outputtedas a process 1, a process of the audio data or the like is executed as aprocess 2 and, after that, the process 3 is started. However, theprocess 3 cannot be completed for a period of time until time t4.

In such a case, hitherto, the image of the frame which has beenoutputted at just previous time t2 and whose management ID is set to Bis outputted again at time t3. However, in the invention, after theprocess of the process 3 was temporarily interrupted at time t3, animage of the frame which has been stored in the output FIFO 4 and whosemanagement ID is set to C is outputted as a process 1. Subsequently,after the process 2 was further executed, when it is finished, theprocess which was once interrupted is restarted as a process 3. Afterthat, subsequently, a process of the video data of the next frame isexecuted. In the example of FIG. 4, the video data of two frames whosemanagement IDs are set to E and F can be processed for a period of timefrom time t3 to time t4, and E and F as management IDs of those framesare stored in the output FIFO 4.

As mentioned above, by enabling the buffers 1 to N to be managed by theoutput FIFO 4, even in the case where the execution time of the process3 temporarily exceeds the output period or in the case where theexecution timings of the process 1 and process 3 become asynchronous dueto the reduction and extension of the execution time of each process,the buffer to which the data is outputted at a predetermined period canbe always assured in the process 1.

As mentioned above, the processing ability of the software that isrequired for the CPU 1 does not need to be made to correspond to thecase where the processing time that is required for decoding of oneframe becomes the longest time. It is sufficient to make it tocorrespond to the average processing time that is required for decodingof a few frames. Therefore, a cheaper processor can be used as a CPU 1.

A digital signal processing apparatus for encoding by the softwareprocess will now be described. FIG. 7 is a block diagram showing aninternal construction of the digital signal processing apparatus towhich the invention is applied. A CPU (Central Processing Unit) 11executes predetermined processes in accordance with a program stored ina memory 12 comprising an ROM (Read Only Memory), an RAM (Random AccessMemory), or the like. A video frame buffer 13 temporarily stores videodata inputted via an input/output interface 14. An encoder 15 encodesthe video data stored in the video frame buffer 13 and outputs it to acode buffer 16. The code buffer 16 temporarily stores the encoded videodata. The video data stored in the code buffer 16 is outputted toanother apparatus, for example, a recording apparatus (not shown)through the input/output interface 14. Those sections are mutuallyconnected by a bus 17.

The video frame buffer 13 has a capacity which can store video data of aplurality of frames and a plurality of buffers have been defined asshown in FIG. 8. That is, N buffers 13-1 to 13-N have been defined inthe video frame buffer 13. Video data 1 is stored in the buffer 13-1,the video data 2 is stored in the buffer 13-2, and the video data N isstored in the buffer 13-N, respectively. The buffers 13-1 to 13-N aremanaged by IDs or pointers for unconditionally specifying them.Explanation will be made hereinbelow on the assumption that they aremanaged by the IDs. The buffers 1 to N are not always necessary to bedefined in continuous address areas.

A writing process of the video data into the video frame buffer 13 whichis executed by the CPU 11 will now be described with reference to aflowchart of FIG. 9. In step S31, when the video data is inputted to anencoding apparatus 10 through the input/output interface 14, the CPU 11examines an empty buffer in the video frame buffer 13 in step S32. Eachbuffer in the video frame buffer 13 is managed by the ID which can beunconditionally specified. The CPU 11 examines the empty buffer byreferring to the ID.

That is, for example, the IDs of the buffers in which the video data hasbeen stored are sequentially written by using an FIFO (First In FirstOut) and by examining the ID which is not written in the FIFO, the emptybuffer can be examined. In the case where each buffer in the video framebuffer 13 is managed by using the FIFO as mentioned above, the data issequentially outputted in the writing order in a reading process, whichwill be explained hereinlater. Since the same ID is not written in theFIFO, the outputted ID (therefore, the ID which is not written in theFIFO) is the ID showing the empty buffer.

In step S33, the video data inputted in step S31 is written in thebuffer examined to be the empty buffer in step S32. The video datawritten in this manner is read out when the encoding is performed by theencoder 15. The reading process will be described with reference to aflowchart of FIG. 10. When the video data has been stored in the videoframe buffer 13 and the CPU 11 is not performing another process, anencoding amount in the case where the image of one frame stored in thevideo frame buffer 13 was encoded is predicted in step S41. The videodata as a target of the encoding is video data stored in the buffercorresponding to the ID which has been written first in the FIFO at thattime point.

When the encoding amount is predicted in step S41, whether the emptycapacity enough to store the data of the encoding amount remains in thecode buffer 16 or not is discriminated in step S42. If it is determinedthat the empty capacity enough to store the data of the predictedencoding amount does not exist in the code buffer 16, the encoded videodata cannot be stored, so that the processes of this flowchart arefinished.

If it is determined in step S42 that the capacity enough to store thedata of the predicted encoding amount exists in the code buffer 16, stepS43 follows. The video data whose encoding amount has been predicted instep S41 is read out from the video frame buffer 13 and encoded by theencoder 15. The video data encoded by the encoder 15 is stored in thecode buffer 16. When the video data is stored in the code buffer 16, thevideo data encoded in step S43 is deleted from the video frame buffer 13in step S44. At the same time, the ID of the buffer in the video framebuffer 13 in which the encoded video data has been stored is abandoned(outputted) from the FIFO.

The encoding by the encoder 15 is executed when the CPU 11 does notperform another process, when the video data has been stored in thevideo frame buffer 13, and when the empty capacity enough to store theencoded video data remains in the code buffer 16 as mentioned above.

A series of processes of the encoding apparatus 10 as mentioned abovewill be further explained with reference to FIGS. 11 and 12. When videodata (a) is inputted to the encoding apparatus 10 at time t, theprocesses described with reference to the flowchart of FIG. 9 areexecuted. The inputted video data (a) is stored in the allocated bufferin the video frame buffer 13. When video data (b) is inputted at timet+1, the inputted video data (b) is stored in the allocated buffer inthe video frame buffer 13 by a process similar to that in the case wherethe video data (a) is inputted. At time t+1, the process in step S41 inthe flowchart of FIG. 4B is executed, namely, the encoding amount at thetime of encoding the video data (a) is predicted with respect to thevideo data (a) stored in the video frame buffer 13. A time intervalbetween time t and time t+1 is set to, for example, 33.36 msec in caseof the NTSC system. The other time intervals are also the same as it.

When it is determined by the process in step S42 that the data of theencoding amount at the time when the video data (a) has been encoded canbe stored in the code buffer 16, the encoding by the encoder 15 isstarted. In this case, since nothing is stored in the code buffer 16, itis determined that there is the empty capacity enough to store theencoded video data (a). When video data (c) is inputted and stored inthe video frame buffer 13 at time t+2, the CPU 1 allows the encoder 5 toencode the video data (a). As a process in step S43, when the video data(a) is read out from the video frame buffer 13, encoded by the encoder15, and stored in the code buffer 16, as a process in step S44, thevideo data (a) (ID corresponding to the buffer in which the video data(a) has been stored) stored in the video frame buffer 13 is deleted.

At time t+2, there is still a time until next time t+3 at a time pointwhen the storage of the video data (c) and the encoding of the videodata (a) are finished and an empty capacity enough to store the encodedvideo data (b) remains in the code buffer 16. Therefore, not only theencoding of the video data (a) but also the encoding of the video data(b) are started. However, when input time t+3 of video data (d) comesand the video data (d) is inputted during the encoding of the video data(b), the encoding of the video data (b) is interrupted while the videodata (d) is being inputted (stored). When the input of the video data(d) is finished, the encoding of the video data (b) is restarted. Theencoding of the video data (b) is finished and the encoding of the videodata (c) is also completed until time t+4. Therefore, just before timet+4, a state where only the video data (d) newly inputted has beenstored in the video frame buffer 13 is obtained, and a state where thevideo data (a), video data (b), and video data (c) which had alreadybeen encoded have been stored in the code buffer 16 is obtained.

As mentioned above, when the video data stored in the video frame buffer13 is encoded, its encoding amount differs every video data. Therefore,even in the case where the time which is required for encoding alsodiffers, there is no need to finish the process within a predeterminedprocessing time (in the input period in which the video data isinputted), so that the encoding precision can be raised.

Video data (e) is inputted and stored in the video frame buffer 13 attime t+4. At this time point, since the video data (d) has already beenstored in the video frame buffer 13 and the CPU 1 does not performanother process, it can execute the encoding process. However, since theempty capacity in the code buffer 16 is not enough, the encoding is notperformed. Video data (f) is inputted at time t+5 (FIG. 12) and apredetermined amount of video data stored in the code buffer 16 iscontinuously outputted to another apparatus, for example, a recordingapparatus or the like. In this output process, since it is sufficientthat the CPU 11 instructs only the start of the reading process byproviding a DMAC (Direct Memory Access Controller) or the like andallowing the data to be DMA transferred, the video data can be outputtedwithout tasking the CPU 1 itself.

Since the video data stored in the code buffer 16 is outputted, an emptycapacity is produced in the code buffer 16 itself. Therefore, theencoding is restarted at time t+6. After that, as described withreference to the flowcharts of FIGS. 9 and 10, the storing process tothe video frame buffer 13 and the encoding process are similarlyexecuted.

As mentioned above, the encoding process for one video data is executedin consideration of the empty capacity in the code buffer 6 and theempty time of the CPU 11, thereby enabling the encoding process to beperformed without being restricted by a condition such that the processhas to be finished within the input period of the video data. Thus, evenif the processor (CPU 11) of the low processing ability is used in theencoding apparatus 10, the encoding can be performed at high precisionand both of the input and the encoding of the video data can beperformed.

A medium which is used for installing a program for executing theforegoing series of encoding or decoding processes into a computer andsetting the program into a state where it can be executed by thecomputer will now be described with reference to FIG. 13.

As shown in FIG. 13A, the program can be provided for the user in astate where it has previously been installed in a hard disk 22 or asemiconductor memory 23 (corresponding to the memory 2) as a recordingmedium built in a personal computer 21.

Otherwise, as shown in FIG. 13B, the program can be temporarily orpermanently stored in a recording medium such as floppy disk 31, CD-ROM32, MO disk 33, DVD 34, magnetic disk 35, semiconductor memory 36, orthe like and provided as package software.

Further, as shown in FIG. 13C, the program can be transferred from adownloading site 41 to a personal computer 43 through a satellite 42 ina wireless manner, or can be transferred to the personal computer 43 ina wired or wireless manner through a network 51 such as local areanetwork or Internet and can downloaded to a built-in hard disk or thelike in the personal computer 43.

The “medium” in the specification denotes a broad concept including allof those media.

Although the access unit has been set to the frame in the abovedescription, it can be also set to a field. Although the case ofdecoding the digital signal encoded by the MPEG system has been shown asan example, the encoding (compression) and its decoding (decompression)process can be also obviously performed by other systems.

As mentioned above, according to the digital signal processing apparatusaccording to claim 1, the digital signal processing method according toclaim 3, and the providing medium according to claim 4, the storedoutputting order of the digital signal of the access units is managed bythe FIFO format. Therefore, the decoding process can be performed by aninexpensive apparatus while keeping the real-time performance by thesoftware.

According to the digital signal processing apparatus according to claim5, the digital signal processing method according to claim 6, and themedium acccording to another embodiment of the invention, the dataamount at the time when the inputted video data is encoded is predicted,the video data is encoded, the encoded video data is stored, theencoding is performed when it is determined that the data of thepredicted amount can be stored, and the encoding is interrupted whilethe video data is being inputted and processed. Therefore, the encodingcan be performed at high precision even in case of using the processorof the low processing ability.

The invention is not limited to the foregoing embodiments or the likebut many modifications and variations are possible within the scope ofclaims without departing from the spirit of the present invention.

1. A digital signal processing method, comprising: inputting an encodeddigital video signal and an encoded digital audio signal; accessing adecoding program for decoding the encoded digital video signal and theencoded digital audio signal from a program memory; executing thedecoding program to decode the encoded digital video signal and theencoded digital audio signal in a decoding operation to thereby generatea decoded digital video signal and a decoded digital audio signal as aplurality of frames; storing the decoded digital video signal in abuffer as data corresponding to the plurality of frames; storingmanagement data in a FIFO format, the management data indicating anoutput order of the plurality of frames; and controlling the decodingoperation based on the output order of the plurality of frames; whereindecoding of the encoded digital audio signal is set to a higher prioritythan decoding of the encoded digital video signal; and wherein if thedecoding operation on the encoded digital video signal is not completedwithin one frame period, the decoding operation is continued in thefollowing frame period.
 2. A computer readable storage medium storing acomputer program for causing a digital video signal processing apparatusto execute a decoding operation, the program comprising: inputting anencoded digital video signal and an encoded digital audio signal;accessing a decoding program for decoding the encoded digital videosignal and the encoded digital audio signal from a program memory;executing the decoding program to decode the encoded digital videosignal and the encoded digital audio signal in the decoding operation tothereby generate a decoded digital video signal and a decoded digitalaudio signal as a plurality of frames; storing the decoded digital videosignal in a buffer as data corresponding to a plurality of frames;storing management data in a FIFO format, the management data indicatingan output order of the plurality of frames; and controlling the decodingoperation based on the output order of the plurality of frames; whereindecoding of the encoded digital audio signal is set to a higher prioritythan decoding of the encoded digital video signal; and wherein if thedecoding operation on the encoded digital video signal is not completedwithin one frame period, the decoding operation is continued in thefollowing frame period.
 3. A digital video signal processing apparatus,comprising: means for inputting an encoded digital video signal and anencoded digital audio signal; means for storing a decoding program fordecoding the encoded digital video signal and the encoded digital audiosignal; processing means operable to execute the stored decoding programto decode the encoded digital video signal and the encoded digital audiosignal supplied from the inputting means and to generate a decodeddigital video signal and a decoded digital audio signal as a pluralityof frames in a decoding operation; buffer means for storing the decodeddigital video signal as data corresponding to the plurality of frames;FIFO means for storing management data indicating an output order of theplurality of frames; and management means for controlling the decodingoperation of the processing means based on the output order of theplurality of frames; wherein the decoding for the encoded digital audiosignal is set to a higher priority than the decoding for the encodeddigital video signal; and wherein if the decoding operation on theencoded digital video signal is not completed within one frame period,the decoding operation is continued in the following frame period.
 4. Adigital signal processing apparatus according to claim 3, wherein thedecoding program operates on the digital video signal according to anMPEG process.
 5. The digital video signal processing apparatus of claim3, wherein the buffer means comprises a plurality of buffer areas, andthe management data indicating the output order of the plurality offrames comprises management identifiers of the plurality of bufferareas.
 6. The digital video signal processing apparatus of claim 5,wherein the management data is stored in the FIFO means in the outputorder of the plurality of frames.